# ChatGPT Deployment

***Copyright © Quectel Wireless Solutions Co., Ltd. 2026. All rights reserved.***

---

## **ChatGPT Overview**

ChatGPT is a conversational large language model product from OpenAI based on the GPT (Generative Pre-trained Transformer) architecture. Representative models include GPT-3.5-turbo, GPT-4, GPT-4o, GPT-4o-mini, etc. It supports multi-turn conversation, code generation, logical reasoning, multimodal understanding, and is currently one of the most widely adopted commercial LLMs.

This document demonstrates how to deploy ChatGPT locally on the **Quectel Pi M1** smart controller.

### **Deployment Resource Overview**

Since ChatGPT is a closed-source cloud service, "deploying ChatGPT" essentially means **running a local service gateway on the device that calls the OpenAI API**, so the device and any clients on its local network can perform Q&A through this gateway.

## **ChatGPT Model Overview**

Mainstream models currently available through the OpenAI API:

| **Model** | **Context Window** | **Description** | **Use Case** |
| --- | --- | --- | --- |
| `gpt-4o` | 128K | Latest multimodal flagship model; supports text and image input | Complex reasoning, multimodal |
| `gpt-4o-mini` | 128K | Smaller 4o variant; fast and low-cost | General conversation (recommended default) |
| `gpt-4-turbo` | 128K | GPT-4 long-context version | Long document analysis |
| `gpt-4` | 8K/32K | Classic GPT-4 | High-quality reasoning |
| `gpt-3.5-turbo` | 16K | Veteran budget model | Low-cost simple tasks |

> For embedded-device scenarios, `gpt-4o-mini` is recommended as the default for the best balance of quality, speed, and cost.

## **Deployment Model Performance Metrics**

Since the deployment uses API calls, no model weights are loaded locally. The performance metrics mainly depend on the network and the server side:

| **Metric** | **Typical Value** | **Notes** |
| --- | --- | --- |
| Local memory usage | ~30–50 MB | Python process + dependencies |
| Local CPU usage | < 5% | Request forwarding only |
| Time to first token (TTFT) | 0.5–2 s | Depends on network and server load |
| Generation speed | 30–80 tokens/s | Generated server-side; limited by network |
| Max single input | 128K tokens (4o series) | Limited by model context window |
| Concurrency | Limited by API rate limits (RPM/TPM) | Depends on account tier |
| Offline availability | Not available | Internet connection required |

## **Installation**

### **Dependency Installation**

The ChatGPT gateway service depends on Python 3 and the official `openai` library. M1 comes preinstalled with Python 3.13.5 and pip3, so only Python dependencies need to be installed.

```bash
# 1. Verify Python and pip
python3 --version    # Expected: Python 3.13.5
pip3 --version

# 2. Upgrade pip and install the official openai library (API client) and flask (local gateway)
pip3 install --upgrade pip
pip3 install openai flask

# 3. Verify installation
python3 -c "import openai; print('openai', openai.__version__)"
```

> If pip install fails with a certificate error, add `--trusted-host pypi.org --trusted-host files.pythonhosted.org`.
> For offline installation, run `pip download openai flask -d ./wheels` on a host with internet access, then `adb push` the wheels directory to the device and run `pip3 install --no-index --find-links=./wheels openai flask`.

### **Tool Installation**

Install the curl tool:

```bash
apt install curl
curl --version | head -1    # Verify curl is available
```

If you want to run streaming visualization tests on the device, optionally install `jq` (for JSON processing):

```bash
apt-get install -y jq       # The device runs Debian; apt is available
```

### **Model Selection**

**Important note:** ChatGPT (GPT-4 / GPT-3.5, etc.) is a closed-source commercial model from OpenAI. **The model weights cannot be downloaded, nor can they be run locally.** Therefore, in this section "Model Selection" refers to **choosing which OpenAI API model to call**, not downloading a model file.

Selection recommendations:

- **`gpt-4o-mini`(default recommendation)** — Fast, low-cost, supports 128K context, suitable for most conversational scenarios on embedded devices.

- **`gpt-4o`(high-quality reasoning)** — For complex logic or code tasks; higher latency and cost.

- **`gpt-3.5-turbo`(low-cost simple tasks)** — Legacy budget model.

If you need to run a GPT-style model **fully offline**, consider an open-source alternative (such as GPT-2 117M, Qwen2.5-0.5B, etc.), loaded as GGUF files via llama.cpp on the device. These options do not require a network but lag behind ChatGPT in quality.

## **Launch**

### **Background Deployment and Configuration**

The following deploys an **OpenAI-compatible local gateway service** under `/data/chatgpt/` on the device. It exposes an HTTP interface, forwards requests to the OpenAI API, and supports multi-turn conversation as well as streaming output.

**Directory structure:**

```
/data/chatgpt/
├── config.env          # Configuration file (API Key, model, port, etc.)
├── chatgpt_server.py   # Gateway service main program
└── chatgpt.service     # systemd unit file (persistent background service)
```

**Step 1: Create the directory and configuration file**

```bash
mkdir -p /data/chatgpt
cat > /data/chatgpt/config.env <<'EOF'
# ===== ChatGPT Gateway Configuration =====
# OpenAI API Key (required; replace with your own Key)
export OPENAI_API_KEY="sk-paste-your-openai-key-here"

# API base URL: official by default; change if you use a relay/proxy service
# Official: https://api.openai.com/v1
# Proxy example: https://your-proxy.example.com/v1
export OPENAI_BASE_URL="https://api.openai.com/v1"

# Default model
export CHATGPT_MODEL="gpt-4o-mini"

# Gateway bind address and port (0.0.0.0 allows LAN access)
export SERVER_HOST="0.0.0.0"
export SERVER_PORT="8000"

# System prompt (defines the assistant persona)
export SYSTEM_PROMPT="You are an intelligent assistant running on an embedded device. Please answer concisely and professionally."

# Sampling parameters
export TEMPERATURE="0.7"
export MAX_TOKENS="1024"
EOF
chmod 600 /data/chatgpt/config.env   # Protect the secret
```

**Step 2: Write the gateway service main program**

```bash
cat > /data/chatgpt/chatgpt_server.py <<'PYEOF'
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""ChatGPT Gateway Service: exposes an HTTP interface and forwards to the OpenAI API.
   Two endpoints are provided:
     /v1/chat/completions  -- OpenAI-compatible endpoint (callable by any OpenAI client)
     /ask                  -- Minimal Q&A endpoint (GET/POST, easy for embedded calls)
"""
import os, json, uuid
from flask import Flask, request, Response, jsonify
from openai import OpenAI

# ---- Load configuration ----
CFG_PATH = "/data/chatgpt/config.env"
if os.path.exists(CFG_PATH):
    for line in open(CFG_PATH, encoding="utf-8"):
        line = line.strip()
        if line and not line.startswith("#") and "=" in line and line.startswith("export "):
            k, v = line[len("export "):].split("=", 1)
            os.environ.setdefault(k.strip(), v.strip().strip('"').strip("'"))

API_KEY     = os.environ["OPENAI_API_KEY"]
BASE_URL    = os.environ.get("OPENAI_BASE_URL", "https://api.openai.com/v1")
MODEL       = os.environ.get("CHATGPT_MODEL", "gpt-4o-mini")
HOST        = os.environ.get("SERVER_HOST", "0.0.0.0")
PORT        = int(os.environ.get("SERVER_PORT", "8000"))
SYS_PROMPT  = os.environ.get("SYSTEM_PROMPT", "You are a helpful assistant.")
TEMPERATURE = float(os.environ.get("TEMPERATURE", "0.7"))
MAX_TOKENS  = int(os.environ.get("MAX_TOKENS", "1024"))

client = OpenAI(api_key=API_KEY, base_url=BASE_URL)
app = Flask(__name__)

# In-memory session store (for demonstration; use persistent storage in production)
sessions = {}

def build_messages(sid, user_msg):
    if sid not in sessions:
        sessions[sid] = [{"role": "system", "content": SYS_PROMPT}]
    sessions[sid].append({"role": "user", "content": user_msg})
    return sessions[sid]

@app.route("/ask", methods=["POST"])
def ask_simple():
    """Minimal endpoint: POST /ask  body={"q":"your question","sid":"optional session id"}"""
    data = request.get_json(force=True, silent=True) or {}
    q   = data.get("q", "").strip()
    sid = data.get("sid", "default")
    if not q:
        return jsonify({"error": "missing 'q'"}), 400
    try:
        msgs = build_messages(sid, q)
        resp = client.chat.completions.create(
            model=MODEL, messages=msgs,
            temperature=TEMPERATURE, max_tokens=MAX_TOKENS)
        ans = resp.choices[0].message.content
        sessions[sid].append({"role": "assistant", "content": ans})
        return jsonify({"answer": ans, "model": MODEL})
    except Exception as e:
        return jsonify({"error": str(e)}), 500

@app.route("/v1/chat/completions", methods=["POST"])
def chat_completions():
    """OpenAI-compatible endpoint; supports stream."""
    body = request.get_json(force=True, silent=True) or {}
    msgs = body.get("messages", [])
    stream = body.get("stream", False)
    model  = body.get("model", MODEL)
    # Auto-prepend system prompt if absent
    if not msgs or msgs[0].get("role") != "system":
        msgs = [{"role": "system", "content": SYS_PROMPT}] + msgs
    try:
        if not stream:
            r = client.chat.completions.create(
                model=model, messages=msgs,
                temperature=body.get("temperature", TEMPERATURE),
                max_tokens=body.get("max_tokens", MAX_TOKENS))
            return jsonify(r.model_dump())
        # Streaming
        def gen():
            r = client.chat.completions.create(
                model=model, messages=msgs, stream=True,
                temperature=body.get("temperature", TEMPERATURE),
                max_tokens=body.get("max_tokens", MAX_TOKENS))
            for chunk in r:
                yield f"data: {json.dumps(chunk.model_dump(), ensure_ascii=False)}\n\n"
            yield "data: [DONE]\n\n"
        return Response(gen(), mimetype="text/event-stream")
    except Exception as e:
        return jsonify({"error": {"message": str(e)}}), 500

@app.route("/v1/models", methods=["GET"])
def list_models():
    """Return the list of available models (OpenAI-compatible)."""
    try:
        return jsonify(client.models.list().model_dump())
    except Exception as e:
        return jsonify({"error": str(e)}), 500

@app.route("/health", methods=["GET"])
def health():
    return jsonify({"status":"ok","model":MODEL,"base_url":BASE_URL})

if __name__ == "__main__":
    print(f"[ChatGPT Gateway] model={MODEL} base={BASE_URL} listen={HOST}:{PORT}")
    app.run(host=HOST, port=PORT, threaded=True)
PYEOF
chmod +x /data/chatgpt/chatgpt_server.py
```

**Step 3: Configure the systemd background service (auto-start on boot)**

```bash
cat > /data/chatgpt/chatgpt.service <<'EOF'
[Unit]
Description=ChatGPT Gateway Service on SC200U
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
WorkingDirectory=/data/chatgpt
EnvironmentFile=/data/chatgpt/config.env
ExecStart=/usr/bin/python3 /data/chatgpt/chatgpt_server.py
Restart=on-failure
RestartSec=5
StandardOutput=append:/data/chatgpt/chatgpt.log
StandardError=append:/data/chatgpt/chatgpt.log

[Install]
WantedBy=multi-user.target
EOF

# Install and enable the service
cp /data/chatgpt/chatgpt.service /etc/systemd/system/chatgpt.service
systemctl daemon-reload
systemctl enable chatgpt.service
```

### **Starting the ChatGPT Service**

**You must configure the API Key before the first start:**

```bash
# Edit the configuration and fill in your real Key
vi /data/chatgpt/config.env
# Modify OPENAI_API_KEY="sk-your-real-key"
```

**Start and management:**

```bash
# Start the service
systemctl start chatgpt.service

# Check status
systemctl status chatgpt.service

# View logs
tail -f /data/chatgpt/chatgpt.log

# Restart / Stop
systemctl restart chatgpt.service
systemctl stop chatgpt.service
```

**Manual foreground debug (bypassing systemd):**

```bash
cd /data/chatgpt
source ./config.env
python3 ./chatgpt_server.py
```

After a successful start, the log will show:

```
[ChatGPT Gateway] model=gpt-4o-mini base=https://api.openai.com/v1 listen=0.0.0.0:8000
 * Running on http://0.0.0.0:8000
```

## **Application Demo**

### **Health Check**

```bash
curl http://127.0.0.1:8000/health
# {"model":"gpt-4o-mini","status":"ok","base_url":"https://api.openai.com/v1"}
```

### **Simple Single-Turn Q&A**

```bash
curl -s -X POST http://127.0.0.1:8000/ask \
  -H "Content-Type: application/json" \
  -d '{"q":"Briefly introduce the SC200U chip in one sentence"}' | python3 -m json.tool
```

Expected output:

```json
{
    "answer": "SC200U is a Quectel IoT communication module based on Qualcomm Snapdragon, supporting 4G all-network access and edge intelligence.",
    "model": "gpt-4o-mini"
}
```

### **Calling the Service from an Embedded Program (C / curl)**

A C program or shell script on the device can call the service via curl:

```bash
# One-line shell call
ANSWER=$(curl -s -X POST http://127.0.0.1:8000/ask \
  -H "Content-Type: application/json" \
  -d "{\"q\":\"${QUERY}\"}" | python3 -c "import sys,json;print(json.load(sys.stdin)['answer'])")
echo "$ANSWER"
```

## **Use Cases**

| **Scenario** | **Description** |
| --- | --- |
| **Smart customer-service terminal** | Provide natural-language Q&A on SC200U smart gateways / POS devices to handle user inquiries |
| **Industrial on-site voice assistant** | Combined with ASR/TTS, engineers can verbally ask for equipment troubleshooting advice |
| **Code / script generation** | Generate shell/Python snippets on-site during operations and maintenance |

## **FAQ**

### **Model Too Large**

ChatGPT itself cannot be run locally. However, if the scenario requires **fully offline** operation or **zero API cost**, you can deploy small open-source models on the M1 as alternatives. The minimal feasible options on this device are:

| **Alternative Model** | **Parameters** | **GGUF Quantization** | **Memory Usage** | **Inference Speed (SC200U)** | **Quality** |
| --- | --- | --- | --- | --- | --- |
| GPT-2 | 124M | F16 | ~0.5 GB | ~15–30 tok/s | Low (English short text only) |
| Qwen2.5-0.5B | 0.5B | Q4_K_M | ~0.6 GB | ~8–15 tok/s | Medium |
| DeepSeek-R1-Distill-Qwen-1.5B | 1.5B | Q4_K_M | ~1.6 GB | ~3–8 tok/s | Medium-high (strong reasoning) |
| Qwen2.5-1.5B | 1.5B | Q4_K_M | ~1.6 GB | ~3–8 tok/s | Medium-high |
| Qwen2.5-3B | 3B | Q4_K_M | ~2.6 GB | ~1–3 tok/s | High (requires swap; somewhat slow) |

> 7B and larger models are not viable on a 3.7 GB memory device (will trigger OOM).